Power-efficient compute-in-memory pooling

ABSTRACT

A multiply-and-accumulate (MAC) circuit having a plurality of compute-in-memory bitcells is configured to multiply a plurality of stored weight bits with a plurality of input bits to provide a MAC output voltage. A successive approximation analog-to-digital converter includes a capacitive-digital-to-analog-converter (CDAC) configured to subtract a bias voltage from the MAC output voltage to provide a CDAC output voltage. A comparator compares the CDAC output voltage to a fixed reference voltage.

TECHNICAL FIELD

This application relates to compute-in-memories, and more particularlyto power-efficient compute-in-memory pooling applications.

BACKGROUND

Computer processing of data typically uses a Von Neumann architecture inwhich the data is retrieved from a memory to be processed in anarithmetic-and-logic unit. In computation-intensive applications such asmachine learning, the data flow from and to the memory becomes abottleneck for processing speed. To address this data-movementbottleneck, compute-in-memory architectures have been developed in whichthe data processing hardware is distributed across the bitcells.

SUMMARY

In accordance with a first aspect of the disclosure, a system for amachine learning application is provided that includes: a firstmultiply-and-accumulate (MAC) circuit including a first plurality ofcompute-in-memory bitcells configured to multiply a plurality of storedweights with an input vector to provide a first MAC output voltage; andan analog-to-digital converter configured to digitize the first MACoutput voltage, the analog-to-digital converter including: a firstcapacitive digital-to-analog converter (CDAC) configured to subtract abias voltage from the first MAC output voltage to provide a first CDACoutput voltage; and a first comparator configured to compare the firstCDAC output voltage to a reference voltage to provide a first comparatoroutput signal.

In accordance with a second aspect of the disclosure, a system for amachine learning application is provided that includes: a firstmultiply-and-accumulate (MAC) circuit including a first plurality ofcompute-in-memory bitcells configured to multiply a first plurality ofstored weights with an input vector to provide a first MAC outputvoltage; a second multiply-and-accumulate (MAC) circuit including asecond plurality of compute-in-memory bitcells configured to multiply asecond plurality of stored weights with the input vector to provide asecond MAC output voltage; and an analog-to-digital converter configuredto digitize either the first MAC output voltage or the second MAC outputvoltage, the analog-to-digital converter including: a first samplingswitch; a second sampling switch; a first capacitive digital-to-analogconverter (CDAC) configured to sample the first MAC output voltagethrough the first sampling switch to provide a sampled first MAC outputvoltage and to subtract a first bias voltage from the sampled first MACoutput voltage to provide a first CDAC output voltage; a secondcapacitive digital-to-analog converter (CDAC) configured to sample thesecond MAC output voltage through the second sampling switch to providea sampled second MAC output voltage and to subtract a second biasvoltage from the sampled second MAC output voltage to provide a secondCDAC output voltage; and a comparator configured to assert a comparatoroutput signal responsive to the first CDAC output voltage being greaterthan the second CDAC output voltage.

In accordance with a third aspect of the disclosure, a system for amachine learning application is provided that includes: amultiply-and-accumulate (MAC) circuit including a plurality ofcompute-in-memory bitcells configured to multiply a plurality of storedweights with a first input vector to provide a first MAC output voltageand to multiply the plurality of stored weights with a second inputvector to provide a second MAC output voltage; and an analog-to-digitalconverter configured to digitize either the first MAC output voltage orthe second MAC output voltage, the analog-to-digital converterincluding: a capacitive digital-to-analog converter (CDAC) configured tosubtract a bias voltage from the first MAC output voltage to provide afirst CDAC output voltage and to subtract the bias voltage from thesecond MAC output voltage to provide a second CDAC output voltage; acomparator configured to compare the first CDAC output voltage to areference voltage to provide a first comparator output signal responsiveto a first assertion of a local enable signal; and a logic gateconfigured to perform a second assertion of the local enable signalresponsive to the first comparator output signal being false, whereinthe comparator is further configured to compare the second CDAC outputvoltage to the reference voltage responsive to the second assertion ofthe local enable signal.

In accordance with a fourth aspect of the disclosure, a system for amachine learning application is provided that includes: amultiply-and-accumulate (MAC) circuit including a plurality ofcompute-in-memory bitcells configured to multiply a plurality of storedweights with a first input vector to provide a first MAC output voltageand to multiply the plurality of stored weights with a second inputvector to provide a second MAC output voltage; and an analog-to-digitalconverter configured to digitize either the first MAC output voltage orthe second MAC output voltage, the analog-to-digital converterincluding: a first sampling switch; a second sampling switch; a firstcapacitive digital-to-analog converter (CDAC) configured to sample thefirst MAC output voltage through the first sampling switch to provide asampled first MAC output voltage; and a second capacitivedigital-to-analog converter (CDAC) configured to sample the second MACoutput voltage through the second sampling switch to provide a sampledsecond MAC output voltage; an averaging switch coupled between the firstCDAC and the second CDAC, wherein the averaging switch is configured toclose to average the sampled first MAC output voltage with the sampledsecond MAC output voltage to provide an averaged MAC output voltage; anda comparator configured to compare the averaged MAC output voltage to areference voltage to provide a comparator output signal.

These and other advantageous features may be better appreciated throughthe following detailed description.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1A illustrates an example multiply-and-accumulate (MAC) circuit forcalculating a filter output in a compute-in-memory machine learningapplication.

FIG. 1B illustrates a conventional ADC for digitizing the output from aMAC circuit in a compute-in-memory machine learning application.

FIG. 2 illustrates a successive approximation ADC for digitizing theoutput from a MAC circuit in a compute-in-memory machine learningapplication in accordance with an aspect of the disclosure.

FIG. 3 illustrates an array of filters (CiM MAC circuits), a Maxoutcircuit, and a Maxpool circuit in accordance with an aspect of thedisclosure.

FIG. 4 illustrates a one-bit Maxout 2 circuit for processing the MACoutput voltages from a pair of filters in accordance with an aspect ofthe disclosure.

FIG. 5 illustrates a multi-bit Maxout 2 circuit for processing the MACoutput voltages from a pair of filters in accordance with an aspect ofthe disclosure.

FIG. 6 illustrates a binary Maxpool 4 circuit for processing fourconsecutive MAC output voltages from a filter in accordance with anaspect of the disclosure.

FIG. 7 illustrates a multi-bit Maxpool 4 circuit for processing fourconsecutive MAC output voltages from a filter in accordance with anaspect of the disclosure.

FIG. 8 illustrates a multi-bit Avgpool 4 circuit for processing fourconsecutive MAC output voltages from a filter in accordance with anaspect of the disclosure.

Embodiments of the present disclosure and their advantages are bestunderstood by referring to the detailed description that follows. Itshould be appreciated that like reference numerals are used to identifylike elements illustrated in one or more of the figures.

DETAILED DESCRIPTION

In deep learning and other machine learning applications, aconvolutional layer is a fundamental building block. A convolutionallayer includes a collection of nodes for the multiplication of filterweights with an input vector from a previous layer (or from input datasuch as an image being analyzed). Each node stores a correspondingfilter weight bit that is multiplied by a corresponding input bit fromthe input vector. Since each node in a convolutional layer is performinga multiplication of a binary filter weight with a corresponding binaryinput, it is convenient to implement each node for a convolutional layerusing a corresponding compute-in-memory bitcell. Should a filter weightbe a multi-bit filter weight, the corresponding node for the filterweight would include a plurality of compute-in-memory bitcells forstoring the bits for the multi-bit filter weight.

Within a convolutional layer, a collection of compute-in-memory bitcellsthat are organized into a compute-in-memory (CiM)multiply-and-accumulate (MAC) circuit are designated as a filter. Theoutput of the CiM MAC circuit represents the multiplication of thestored filter weight bits with the corresponding input bits for thefilter. For example, suppose the filter has the dimensions of 2 by 2 by2. There are thus eight filter weights in such a filter that aremultiplied by the corresponding inputs. The resulting CiM MAC circuitperforms eight multiplications and sums them to form a MAC outputvoltage that is digitized to provide an output signal that is propagatedto the next layer.

A particularly advantageous analog CiM MAC circuit is disclosed thatprovides a rail-to-rail (ground to the power supply voltage) filteroutput for low-power deep learning and other artificial intelligenceapplications. As compared to a digital implementation, an analog CiM MACarchitecture consumes less power. But the analog output from an analogCiM MAC circuit needs to be digitized by an analog-to-digital converter(ADC) that would conventionally require a full-rail comparator. As knownin the digital electronic arts, a comparator functions to compare twoanalog input signals to each other. Depending upon which input signal isgreater, a comparator either asserts its output signal to the powersupply voltage VDD or to ground. The power supply voltage VDD is carriedon a power supply lead or conductor commonly referred to as a powersupply rail. If an input signal voltage can range from ground to thepower supply voltage VDD, such an input signal is a “full-rail” inputsignal since it can range from ground to the rail voltage. If both inputsignals to the comparator are full-rail signals, the design for such afull-rail comparator is more demanding as contrasted to a comparator inwhich one of the input signals has a fixed mid-range voltage (e.g.,VDD/2). An ADC is provided herein that advantageously eliminates theneed for a full-rail comparator as will be further explained herein.Note that the analog compute-in-memory bitcells in the analog CiM MACare more complex than a corresponding conventional bitcell such as asix-transistor static-random-access-memory (SRAM) bitcell since acompute-in-memory bitcell not only stores its filter weight but alsoimplements a logic gate such as an exclusive-not-OR (XNOR) or anexclusive-OR (XOR) gate for the multiplication of the stored filterweight with the corresponding input bit. The storage of the filterweight in the compute-in-memory bitcell may be implemented using eitheran SRAM or a dynamic random access memory (DRAM) architecture. Aparticularly advantageous compute-in-memory SRAM bitcell will bediscussed further herein. But it will be appreciated that thecompute-in-memory architectures disclosed herein are widely applicableto any suitable compute-in-memory bitcell implementation including aDRAM CiM bitcell architecture. It will thus be appreciated that thefollowing claims are not limited to SRAM bitcell implementations unlesssuch a limitation is expressly stated.

Each compute-in-memory SRAM bitcell stores a filter weight bit using twocross-coupled inverters. An example CiM MAC circuit 100 is shown in FIG.1A that includes a plurality of N CiM bitcells ranging from a firstbitcell 105 to an Nth bitcell 110 but it will be appreciated that thenumber N of CiM bitcells will depend upon the filter dimensions. Theterms “filter” and “MAC circuit” are used interchangeably herein. Eachcompute-in-memory SRAM bitcell is substantially identical so thefollowing discussion for bitcell 105 is equally applicable to all theremaining bitcells in CiM MAC circuit 100. Bitcell 105 includes a pairof cross-coupled inverters that store the filter weight bit on a trueoutput node Q and also store a complement of the bit on a complementoutput node QB. As known in the SRAM arts, the filter weight bit forbitcell 105 was written from a bit line BL and a complement bit line BLBwhen a first write word line (WWL1) is asserted to a power supplyvoltage VDD to switch on a corresponding pair of NMOS access transistorsM1 and M2. The true output node Q connects to a source of a p-typemetal-oxide semiconductor (PMOS) first pass transistor P1 that has itsdrain connected to a second plate of a capacitor C and to a drain of ann-type metal-oxide semiconductor (NMOS) reset transistor M3. Similarly,the complement output node QB connects to a source of a PMOS second passtransistor P2 that has its drain connected to the second plate ofcapacitor C and to the drain of reset transistor M3. An active-low inputvector bit carried on a first pre-charge word line PCWL1 controls thegate of first pass transistor P1. Similarly, a complement of theactive-low input vector bit carried on a first complement pre-chargeword line PCWLB1 controls the gate of second pass transistor P2. Forbrevity, a pre-charge word line such as PCWL1 is also denoted as a wordline herein.

A first plate of capacitor C connects to a read bit line RBL thatextends across all the bitcells in CiM MAC circuit 100. Prior to acalculation phase, the capacitor C is reset in a reset phase forcompute-in-memory bitcell 105. During the reset phase, an active-lowreset signal carried on a reset line is asserted to switch on a PMOStransistor P4 connected between the read bit line RBL and a node for thepower supply voltage VDD. The read bit line is thus charged to the powersupply voltage VDD during the reset phase. While the reset signal isasserted, a read word line (RWL) is also asserted that connects to agate of reset transistor M3. A source of reset transistor M3 isconnected to ground so that when the read word line is asserted, resettransistor M3 switches on to ground the second plate of capacitor C. Thecapacitor C is thus charged to the power supply voltage VDD during thereset phase. During the reset phase, both the pre-charge word line PCWL1and the complement pre-charge word line PCWLB1 are charged to the powersupply voltage VDD to maintain both pass transistors P1 and P2 off.

In a calculation phase to calculate the binary multiplication of thestored bit and the input vector bit in bitcell 105, the pre-charge wordline PCWL1 and the complement pre-charge word line PCWLB1 are chargedaccording to the value of the input vector bit for bitcell 105 while thereset signal is asserted to keep the read bit line RBL charged to thepower supply voltage VDD. The read word line RWL is de-asserted duringthe calculation phase so that the second plate of the capacitor C floatswith respect to ground during the calculation phase. In an active-lowembodiment in which a true input vector bit is ground and a false inputvector bit is VDD, the pre-charge word line PCWL1 is discharged if theinput vector bit is true. At the same time, the complement pre-chargeword line PCWLB1 is then charged high to the power supply voltage VDD.Conversely, if the input vector bit is false in an active-lowembodiment, the pre-charge word line PCWL1 is charged to the powersupply voltage VDD while the complement pre-charge word line PCWLB1 isdischarged. If the pre-charge word line PCWL1 is discharged due to thetrue value of the input vector bit and the stored weight bit is alsotrue, pass transistor P1 will switch on to charge the second plate ofthe capacitor C to the power supply voltage VDD. Since the read bit lineRBL is connected to the power supply node for the power supply voltageVDD, the capacitor C is thus discharged due to the charging of itssecond plate. The same discharge for capacitor C occurs when both thefilter weight bit and the input vector bit are false. In that case,second pass transistor P2 switches on to charge the second plate of thecapacitor C during the calculation phase. But if the input vector bitand the stored bit have complementary binary values, the second platethen stays discharged so that the capacitor C remains charged. Theresulting multiplication is thus an XNOR of the input vector bit and thefilter weight bit. On the other hand, the multiplication would be an XORof the input vector bit and the filter weight bit if the input vectorbit is an active-high signal (note that a true active-high signal is thepower supply voltage VDD whereas a false active-high signal is ground).

The same reset phase and calculation phase occurs for the remainingbitcells such as bitcell 110. Each bitcell has its own correspondingpre-charge word line and complement pre-charge word line. For example,bitcell 110 responds to an Nth precharge word line PCWLN and an Nthcomplement pre-charge word line PCWLBN. Similarly, each bitcell iswritten to by its own corresponding write word line. For example, an Nthwrite word line WWLN is asserted to write the corresponding filterweight bit into bitcell 110.

An accumulation phase follows the calculation phase. In the accumulationphase, the read word line RWL is asserted while the active-low resetsignal is de-asserted (charged to VDD). The read bit line RBL is thusisolated during the accumulation phase from the power supply nodebecause transistor P4 switches off. The second plate of the capacitor Cis grounded during the accumulation phase as reset transistor M3 isswitched on due to the assertion of the read word line to the powersupply voltage VDD. If capacitor C had been discharged in bitcell 105,the read bit line voltage will thus be decreased from the power supplyvoltage VDD. Conversely, capacitor C will support the read bit linevoltage if capacitor C had remained charged. The accumulation phase forthe bitcells such as bitcell 110 each occurs at the same time andaffects the read bit line voltage in the same fashion. The outputvoltage of analog CiM MAC circuit 100 is thus the resulting read bitline voltage after the accumulation phase for all its bitcells.

To provide greater flexibility, note that the second plate of thecapacitor C in each bitcell may be charged to the power supply voltageVDD through a corresponding PMOS transistor P3. For example, transistorP3 in bitcell 105 has a source connected to the power supply node VDDand a drain connected to the second plate of capacitor C. The additionof transistor P3 is also advantageous as capacitor C can be reused aspart of a capacitor digital-to-analog converter (CDAC) as discussedfurther herein. After CiM MAC 100 has finished the accumulation phase,the read word line voltage may be sampled by another capacitor (notillustrated). With the sampled voltage captured by this additionalcapacitor, the read bit line may then be discharged to ground. Theresulting sampled voltage may then be selectively boosted by driving thesecond plates of selected ones of capacitors C to the power supplyvoltage VDD by switching on transistors P3 in the selectedcompute-in-memory bitcells in the column. In particular, a DAC signalBTP such as controlled by a finite state machine (not illustrated) isdischarged for the selected compute-in-memory bitcells to boost thesampled voltage. The remaining compute-in-memory bitcells in CiM MACcircuit 100 would float the second plate for their capacitor C so as tonot affect the desired boosting. Alternatively, the sampled voltage maybe selectively decremented by grounding the second plates of selectedones of capacitors C by switching on reset transistors M3 in theselected compute-in-memory bitcells by asserting their DAC signal BTP.

As discussed earlier, it would be conventional for an analog CiM MACcircuit such as CiM MAC circuit 100 to provide its read bit line voltageto an ADC that includes a CDAC and a full-rail comparator. An exampleconventional successive approximation ADC 101 for digitizing the MACoutput voltage from CiM MAC circuit 100 is shown in FIG. 1B. A CDAC 120converts a digital input (not illustrated) into an analog thresholdvoltage (Vth1) that drives an input of a comparator 125. Comparator 125compares the threshold voltage Vth1 to the analog output voltage to makea bit decision for the digitization of the MAC output voltage. The MACoutput voltage can range from ground to a power supply voltage VDD. CDAC120 must then adjust the threshold voltage accordingly to calculate thevarious bits for digitization. For example, a control logic circuit suchas a finite state machine (not illustrated) generates a digital input toCDAC 120 to set the threshold voltage Vth1 to VDD/2 initially. Ifcomparator 125 indicates that the analog output voltage is greater thanVDD/2, ADC 101 sets a first bit of the digital result as stored in a setof registers (not illustrated). The control logic circuit may thencommand CDAC 120 to increase the threshold voltage Vth1 to ¾*VDDfollowed by a comparison of the analog output voltage and the newthreshold voltage to determine another bit of the digital result. Inthis fashion, CDAC 120 may increase or decrease the threshold voltageVth1 as necessary in a binary progressive fashion until all the bits ofthe digital result are determined. But note that comparator 125 mustthen be a “full rail” comparator in that the two signals being comparedrange virtually from ground to the power supply voltage VDD. Such afull-rail comparator consumes a substantial amount of power.

A successive approximation ADC 201 shown in FIG. 2 advantageously doesnot need a full-rail comparator. ADC 201 digitizes a MAC output voltage(Vin1) from a CiM MAC circuit 200. CiM MAC circuit 200 may be arrangedsuch as discussed for CiM MAC circuit 100. ADC 201 digitizes the MACoutput voltage using a CDAC 205 and a comparator 210. But note that theMAC output voltage does not drive the comparator. Instead, the CDAC 205samples the MAC output voltage such as through a sampling switch S1 tocharge a sampling rail R. The sampled MAC output voltage thus functionsas a voltage input to CDAC 205 that in conventional operation would beconverted into the threshold voltage Vth1. CDAC 205 is a 3-bit converterand thus includes a binary progression of capacitors C, 2C, 4C, and 8Cas well as the dummy least-significant bit (LSB) capacitor C. There arethus two capacitors C in CDAC 205. It will be appreciated, however, thatthe CDAC bit precision may be greater than or less than 3 bits inalternative implementations. Each capacitor has a first plate attachedto the sampling rail R and a second plate connected through a pair ofswitches to either a node for the power supply voltage Vdd (denoted asvp) or to ground (denoted as vn). Referring again to MAC 100, note thatthe CiM bitcells may be repurposed to function in CDAC 205 since theswitching on of the bitcell's transistor P3 functions as the switch tovp. Similarly, the switching on of a bitcell's transistor M3 functionsas the switch to vn.

In a binary implementation for ADC 201, it would be conventional forCDAC 205 to adjust its capacitors so that the sampling rail voltageequals the MAC output voltage minus the threshold voltage Vth1 asdiscussed previously. But CDAC 205 instead subtracts a bias voltage(Vth1−Vdd/2) from the MAC output voltage Vin1, where Vdd is the powersupply voltage. Comparator 210 may thus compare the resulting samplingrail voltage to a reference voltage such as one-half of the power supplyvoltage Vdd. Since comparator 210 compares using the fixed voltageVdd/2, the design of comparator 210 is relaxed as compared to aconventional full-rail comparator such as comparator 125. In a binaryembodiment for ADC 201, the output of comparator 210 is the one-bitdigitization of the MAC output voltage Vin1. In a multi-bit embodimentfor ADC 201, a control logic circuit such as a finite state machine(FSM) 215 controls CDAC 205 to adjust Vth1 to calculate the additionalbits in successive comparisons by comparator 210 analogously asdiscussed for CDAC 120. To reset after a conversion, CDAC 205 closes aswitch S2 to discharge the sampling rail R to ground.

An ADC such as ADC 201 may be advantageously used in machine-learningpooling applications. Pooling is a down sampling technique from onelayer to another to reduce the computation burden. There are at leastthree forms of pooling. In a first pooling form denoted herein as amaximum out (Maxout) pooling, it is conventional that the digitaloutputs from two or more filters are compared so that only the greatestdigital output from the filters propagates to the next layer. Forexample, suppose there are 10 filter outputs but only the five greatestones are allowed to propagate to the next layer. In a second poolingform denoted herein as a maximum pooling (Maxpool), the pooling is notacross filters but instead is intra-filter. With regard to any givenfilter, it will process a first input vector and provide a first output,process a second input vector and provide a second output, and so on. Ina Maxpool pooling, only the greatest output from a plurality ofconsecutive outputs over time propagates to the next layer. The downsampling in Maxpool is thus temporal. A third form of pooling is denotedherein as an average pooling (Avgpool). An Avgpool pooling is avariation of Maxpool pooling in that in Avgpool it is the average of thegroup of consecutive outputs that propagates to the next layer. The downsampling in average pooling is thus temporal in that it averages over aseries of input vectors to the filters being pooled.

From the preceding discussion, it can be seen that a Maxout pooling downsamples the filters so that a reduced set of filter outputs propagatesto the next layer. In contrast, the down sampling is temporal in aMaxpool or an Avgpool pooling. Note that a Maxout pooling and a temporaldown sampling (either Maxpool or Avgpool) may be performed on the samelayer. For example, a down sampling of the filters by one-half (Maxout2) followed by a Maxpool down sampling by four (Maxpool 4) is shown inFIG. 3 for a plurality of 10 filters arranged from a first filter(filter 1) to a tenth filter (filter 10). Since it is a down sampling bytwo, there are five outputs from the Maxout 2 pooling. These fiveoutputs are then sampled across four cycles of the input vector to thefilters by the Maxpool 4 down sampling. The resulting five outputs wouldthen propagate to the next layer (not illustrated).

It is conventional for the pooling schemes discussed above to beperformed in the digital domain. The comparator in the ADC associatedwith each filter such as ADC 101 discussed previously must then make thenecessary comparisons. The resulting comparator power consumption is amajor contributor to the overall power consumption by a filter. But thepooling circuits disclosed herein advantageously limit the comparatorpower consumption. The implementation depends upon whether the filteroutput is a one-bit or a multi-bit output. For example, the Maxout 2pooling shown in FIG. 3 is illustrated in greater detail as a one-bitMaxout 2 circuit 400 in FIG. 4. A first filter (filter 1) produces anMAC output voltage vin1 that is sampled by a CDAC 410. Similarly, asecond filter (filter 2) produces a MAC output voltage vin2 that issampled by a CDAC 405. CDAC 405 and 410 are both 3-bit converters butthe precision of the converters may be varied in alternativeimplementations. CDAC 405 and 410 include the capacitors 8C, 4C, 2C, C,and C as well as switches S1 and S2 as discussed for CDAC 205. CDAC 405is configured to subtract the difference between its threshold voltagevth2 and Vdd/2 from the MAC output voltage vin2 from filter 2.Similarly, CDAC 410 is configured to subtract the difference(vth1−Vdd/2) from the MAC output voltage vin1 from filter 1, where vth1is the threshold voltage for CDAC 410. Given this subtraction, thecorresponding comparators may compare their CDAC output voltages to thefixed threshold Vdd/2. Thus, a comparator 420 compares the CDAC outputvoltage from CDAC 410 to Vdd/2 to form a first comparator output signal(Comp1) that is asserted when the difference (vin1−Vth1) is greater thanzero. Similarly, a comparator 415 compares the CDAC output voltage fromCDAC 405 to Vdd/2 to form a second comparator output signal (Comp2) thatis asserted when the difference (vin2−Vth2) is greater than zero.Comparators 415 and 420 may thus each have a simplified design ascompared to a conventional full-rail comparator. Although comparators415 and 420 are thus relatively-low power as compared to the use offull-rail comparators, additional power savings are achieved by timemultiplexing the enabling of comparator 415 and 420. In particular, afirst enable signal (en_ph1) enables comparator 420 whereas a secondenable signal (en_ph2) enables comparator 415. The first enable signalis asserted prior to the assertion of the second enable signal.Comparator 420 will thus perform its comparison prior to the comparisonby comparator 415.

To reduce the comparator power consumption even further, the firstcomparator output signal Comp1 prevents comparator 415 from beingenabled if the first comparator output signal Comp1 is true. As usedherein, a binary signal is deemed to be asserted if its logical state istrue, regardless of whether that assertion is active-high or active-low.To accommodate this blocking, a logic gate such as an AND gate 425performs a logical AND of the second enable signal en_ph2 with acomplement of the first comparator output signal Comp1. If the firstcomparator output signal Comp1 is true, an output of AND gate 425 willthen be false (de-asserted). The output of AND gate 425 drives an enableinput for comparator 415 so that comparator 415 is not enabled if thefirst comparator output signal Comp1 is true. But if the firstcomparator output signal Comp1 is false and the second enable signalen_ph2 is enabled, AND gate 425 enables comparator 415. Comparator 415is thus enabled only if the digitized output from filter 1 (the firstcomparator output signal Comp1) is a logic zero. Both the firstcomparator output signal Comp1 and the second comparator output signalComp2 are ORed in an OR gate 430 to form the Maxout output signal forfilters 1 and 2. When the first comparator output signal Comp1 is true,Maxout circuit 400 thus saves a substantial amount of power sincecomparator 415 is prevented from performing a needless comparison.

An example multi-bit Maxout 2 circuit 500 for producing a multi-bitoutput from a pair of filters is shown in FIG. 5. A first filter(Filter1) produces a MAC output voltage vin1 that is sampled by a CDAC505 through a first sampling switch ph1. Similarly, a second filter(Filter2) produces a MAC output voltage vin2 that is sampled by a CDAC510 through a second sampling switch ph2. A selection logic circuit 520controls the sampling switches ph1 and ph2. To calculate the first bit(e.g., the most-significant bit) for the output from Maxout 2 circuit500, selection logic circuit 520 closes both sampling switches ph1 andph2. CDAC 505 subtracts a second bias voltage (vbias2) from the MACoutput voltage vin2 for filter 2. The second bias voltage is thedifference between the threshold voltage for CDAC 505 and VDD/2.Similarly, CDAC 510 subtracts a first bias voltage (vbias1) from the MACoutput voltage vin1 for filter 1. The first bias voltage is thedifference between the threshold voltage for CDAC 510 and VDD/2. Acomparator 515 is then enabled by an enable signal (en) to compare theresulting output voltages from CDAC 505 and CDAC 510. It is arbitrarywhich filter ultimately drives the positive input or the negative inputof comparator 515 so long as selection logic circuit 520 is configuredaccordingly. In Maxout 2 circuit 500, comparator 515 asserts acomparator output signal (comp) if the CDAC output voltage from CDAC 505is greater than the CDAC output voltage from CDAC 510. Should thecomparator output signal comp indicate that the CDAC output voltage fromCDAC 505 is greater than the CDAC output voltage from CDAC 510,selection logic circuit 520 opens first sampling switch ph1 and closessecond sampling switch ph2. CDAC 510 may then be configured so that itsCDAC output voltage equals a reference voltage such as VDD/2. A finitestate machine 525 functions as discussed for finite state machine 215 tocontrol the second bias voltage vbias2 for CDAC 505 so that comparator515 may successively determine the remaining bits for the digitizationof the MAC output voltage vin2 from filter 2. Note that no power is thenwasted calculating any additional bits for filter 1. Conversely, shouldthe initial comparison by comparator 515 indicate that the CDAC outputvoltage from CDAC 510 was greater than the CDAC output voltage from CDAC505, selection logic circuit 520 would close first sampling switch ph1and open second sampling switch ph2. The remaining bits for thedigitization of the MAC output voltage vin1 would be calculated withoutany power wasted on the digitization of additional bits for thedigitization of the MAC output voltage vin2. Note that a non-linearactivation may then be applied to the digital output from Maxout 2circuit 500 since it is typical that the non-linear activations used inmachine learning applications are monotonically increasing functions.

A binary Maxpool 4 circuit 600 illustrated in FIG. 6 will now bediscussed. A filter 605 successively processes four input vectorsranging from an input vector 1 to an input vector 4 to produce foursuccessive MAC output voltages (vin1). A CDAC 610 functions as discussedfor CDAC 205 to sample the MAC output voltage vin1 through samplingswitch S1 and to subtract a bias voltage equaling the difference of itsthreshold voltage from the sampled MAC output voltage. CDAC 610 resetsthrough switch S2 analogously as discussed for CDAC 205. A comparator615 compares the CDAC output voltage to VDD/2 to make a bit decision asa comparator output signal Comp1. A delay circuit (dly) delays thecomparator output signal to drive an inverting input of an AND gate 620.AND gate 620 receives an enable signal (en) that is asserted for eachsuccessive MAC output voltage digitization. AND gate 620 ANDS the enablewith the delayed inverted comparator output signal. It is only if thecomparator output signal is a binary false signal that an output of ANDgate 620 becomes true. But if the comparator output signal is asserted,the output of AND gate 620 is then de-asserted. To allow for a resetfunction, the output of AND gate 620 is processed in a second AND gatewith a complement reset signal (rstb). With the reset signal notasserted, the complement reset signal rstb is true so that the output ofAND gate 620 propagates through the second AND gate to form a localenable signal (en_local) for comparator 615. Once the comparator outputbecomes true, comparator 615 will thus not be enabled for the remainingCDAC output voltages. After the four input vectors have been processedthrough filter 605, a latch signal is asserted so that a flip-flop 625latches the comparator output signal.

A multi-bit Maxpool 4 circuit 700 is shown in FIG. 7. Maxpool circuit700 is analogous to Maxout circuit 500 except that there is just onefilter 701 that is processing four consecutive input vectors rangingfrom an input vector 1 to an input vector 4. A CDAC 710 samples a MACoutput voltage vin1 for a first one of the input vectors (input vector1) processed by filter 701 through a first sampling switch ph1.Similarly, a CDAC 705 samples the MAC output voltage vin1 for a secondone of the input vectors (input vector 2) processed by filter 701through a second sampling switch ph2. Each CDAC 705 and 710 subtracts abias voltage analogously as discussed for CDACs 505 and 510. Acomparator 715 compares the resulting CDAC output voltages. A selectionlogic circuit 720 controls the sampling switches ph1 and ph2 so thatCDAC having the smallest CDAC output voltage is overwritten with the MACoutput voltage for the third input vector (input vector 3). Comparator715 again determines the largest CDAC output voltage so that the CDACwith the smallest output voltage can again be overwritten with the MACoutput voltage for the fourth input vector (input vector 4). Once thelargest CDAC output voltage is determined, the remaining CDAC may berepurposed analogously as discussed for Maxout 2 circuit 500 to providethe reference voltage VDD/2 to comparator 715. A finite state machine725 controls the threshold voltage in the CDAC with the greatest sampledvoltage to calculate the remainder of the bits for the digitization ofthe MAC output voltage resulting from the corresponding input vector.

An Avgpool 4 circuit 800 is shown in FIG. 8. A filter 805 processes fourinput samples (input vectors) ranging from an input vector 1 to an inputvector 4 to produce four successive samples of a MAC output voltagevin1. Each successive sample of the MAC output voltage vin1 is assignedto a corresponding CDAC through a corresponding sampling switch. Thereis a CDAC1 for sampling the MAC output voltage vin1 from the processingof the input vector 1, a CDAC2 for sampling the MAC output voltage vin1from the processing of the input vector 2, a CDAC3 for sampling the MACoutput voltage vin1 from the processing of the input vector 3, and aCDAC4 for sampling the MAC output voltage vin1 from the processing ofthe input vector 4. When a given one of the CDACs is sampling its MACoutput voltage, the sampling switch for that CDAC is closed while theremaining sampling switches are open. There is a first sampling switchph1 for CDAC1, a second sampling switch ph2 for CDAC2, a third samplingswitch ph3 for CDAC3, and a fourth sampling switch for CDAC4. After eachCDAC has sampled its respective MAC output voltage, the samplingswitches are opened. Each CDAC functions to subtract a bias voltage fromits sampled MAC output voltage analogously as discussed for CDACs 705and 710.

When a particular sampling switch is closed, the other are opened. Forexample, a first sampling switch ph1 closes so that CDAC1 can sample theMAC output voltage (Vin1) resulting from the processing of the firstinput vector. Similarly, the second sampling switch ph2 closes so thatCDAC2 can sample the MAC output voltage resulting from the processing ofthe second input vector. The third sampling switch ph3 then closes sothat CDAC3 can sample the MAC output voltage resulting from theprocessing of the third input vector. Finally, the fourth samplingswitch ph4 closes so that CDAC4 can sample the MAC output voltageresulting from the processing of the fourth input vector.

Once the CDACs have all sampled their MAC output voltages, an averagingphase commences. To perform the averaging, there is a first averagingswitch avg1 extending between the output rail of CDAC1 to the outputrail of CDAC2, a second averaging switch avg2 extending between theoutput rail of CDAC2 to the output rail of CDAC3, and a third averagingswitch avg3 extending between the output rail of CDAC3 to the outputrail of CDAC4. To perform the averaging of the CDAC output voltages, thethree averaging switches avg1, avg2, and avg3 are closed simultaneously.The capacitors in the CDACs (see, e.g., CDAC 610) provide an inherentaveraging function during the averaging phase. At the same time, eachCDAC subtracts its threshold voltage (Vth). The resulting averagedvoltage stored by the CDACs thus equals (Vin1+Vin2+Vin3+Vin4)/4−Vth.

Once the CDAC voltages are averaged and the threshold voltagesubtracted, one of the CDACs (e.g., CDAC1) then functions to digitizethe averaged CDAC voltage with a comparator 810 analogously as discussedwith comparator 515. The corresponding finite state machine is not shownin FIG. 8 for illustration clarity. Another CDAC (e.g., CDAC2) can thenbe repurposed to provide the VDD/2 threshold voltage to comparator 810while the averaged CDAC voltage is digitized.

It will be appreciated that many modifications, substitutions andvariations can be made in and to the materials, apparatus,configurations and methods of use of the devices of the presentdisclosure without departing from the scope thereof. In light of this,the scope of the present disclosure should not be limited to that of theparticular embodiments illustrated and described herein, as they aremerely by way of some examples thereof, but rather, should be fullycommensurate with that of the claims appended hereafter and theirfunctional equivalents.

We claim:
 1. A system for a machine learning application, comprising: afirst multiply-and-accumulate (MAC) circuit including a first pluralityof compute-in-memory bitcells configured to multiply a first pluralityof stored weights with an input vector to provide a first MAC outputvoltage; and an analog-to-digital converter configured to digitize thefirst MAC output voltage, the analog-to-digital converter including: afirst capacitive digital-to-analog converter (CDAC) configured tosubtract a bias voltage from the first MAC output voltage to provide afirst CDAC output voltage; and a first comparator configured to comparethe first CDAC output voltage to a reference voltage to provide a firstcomparator output signal.
 2. The system of claim 1, wherein thereference voltage is approximately one-half of a power supply voltage.3. The system of claim 1, wherein the bias voltage equals a differencebetween a threshold voltage for the first CDAC and the referencevoltage, the analog-to-digital converter further comprising a controllogic circuit configured to control the threshold voltage.
 4. The systemof claim 3, wherein the control logic circuit is a finite state machine.5. The system of claim 1, further comprising: a second MAC circuitincluding a second plurality of compute-in-memory bitcells configured tomultiple a second plurality of stored weights with the input vector toprovide a second MAC output voltage; and wherein the analog-to-digitalconverter further includes: a second CDAC configured to subtract thebias voltage from the second MAC output voltage to provide a second CDACoutput voltage; a first logic gate configured to assert a local enablesignal responsive to the first comparator output signal being false; asecond comparator configured to respond to an assertion of the localenable signal to compare the second CDAC output voltage to the referencevoltage to provide a second comparator output signal; and a second logicgate configured to provide a maximum pooling output signal for the firstMAC circuit and for the second MAC circuit responsive to the firstcomparator output signal and to the second comparator output signal. 6.The system of claim 5, wherein the first logic gate is an AND gateconfigured to AND a complement of the first comparator output signal anda global enable signal to produce the local enable signal.
 7. The systemof claim 5, wherein the second logic gate is an OR gate configured to ORthe first comparator output signal and the second comparator outputsignal to provide the maximum pooling output signal.
 8. A system for amachine learning application, comprising: a firstmultiply-and-accumulate (MAC) circuit including a first plurality ofcompute-in-memory bitcells configured to multiply a first plurality ofstored weights with an input vector to provide a first MAC outputvoltage; a second MAC circuit including a second plurality ofcompute-in-memory bitcells configured to multiply a second plurality ofstored weights with the input vector to provide a second MAC outputvoltage; and an analog-to-digital converter configured to digitizeeither the first MAC output voltage or the second MAC output voltage,the analog-to-digital converter including: a first sampling switch; asecond sampling switch; a first capacitive digital-to-analog converter(CDAC) configured to sample the first MAC output voltage through thefirst sampling switch to provide a sampled first MAC output voltage andto subtract a first bias voltage from the sampled first MAC outputvoltage to provide a first CDAC output voltage; a second capacitivedigital-to-analog converter (CDAC) configured to sample the second MACoutput voltage through the second sampling switch to provide a sampledsecond MAC output voltage and to subtract a second bias voltage from thesampled second MAC output voltage to provide a second CDAC outputvoltage; and a comparator configured to assert a comparator outputsignal responsive to the first CDAC output voltage being greater thanthe second CDAC output voltage.
 9. The system of claim 8, furthercomprising: a selection logic circuit configured to close the firstsampling switch and to open the second sampling switch responsive to anassertion of the comparator output signal; and a control logic circuitconfigured to control the first bias voltage during a digitization ofthe sampled first MAC output voltage by the first CDAC.
 10. The systemof claim 9, wherein the comparator is further configured to compare thefirst CDAC output voltage to a reference voltage during the digitizationof the sampled first MAC output voltage by the first CDAC.
 11. Thesystem of claim 10, wherein the reference voltage is one-half of a powersupply voltage.
 12. A system for a machine learning application,comprising: a multiply-and-accumulate (MAC) circuit including aplurality of compute-in-memory bitcells configured to multiply aplurality of stored weights with a first input vector to provide a firstMAC output voltage and to multiply the plurality of stored weights witha second input vector to provide a second MAC output voltage; and ananalog-to-digital converter configured to digitize either the first MACoutput voltage or the second MAC output voltage, the analog-to-digitalconverter including: a capacitive digital-to-analog converter (CDAC)configured to subtract a bias voltage from the first MAC output voltageto provide a first CDAC output voltage and to subtract the bias voltagefrom the second MAC output voltage to provide a second CDAC outputvoltage; a comparator configured to compare the first CDAC outputvoltage to a reference voltage to provide a first comparator outputsignal responsive to a first assertion of a local enable signal; and alogic gate configured to perform a second assertion of the local enablesignal responsive to the first comparator output signal being false,wherein the comparator is further configured to compare the second CDACoutput voltage to the reference voltage to provide a second comparatoroutput signal responsive to the second assertion of the local enablesignal.
 13. The system of claim 12, wherein the reference voltage isone-half of a power supply voltage.
 14. The system of claim 12, whereinthe logic gate is an AND gate.
 15. The system of claim 12, wherein thelogic gate comprises a pair of AND gates.
 16. The system of claim 12,further comprising a delay circuit to delay the first comparator outputsignal.
 17. The system of claim 12, further comprising a storage deviceconfigured to store the first comparator output signal and the secondcomparator output signal.
 18. A system for a machine learningapplication, comprising: a multiply-and-accumulate (MAC) circuitincluding a plurality of compute-in-memory bitcells configured tomultiply a plurality of stored weights with a first input vector toprovide a first MAC output voltage and to multiply the plurality ofstored weights with a second input vector to provide a second MAC outputvoltage; and an analog-to-digital converter configured to digitizeeither the first MAC output voltage or the second MAC output voltage,the analog-to-digital converter including: a first sampling switch; asecond sampling switch; a first capacitive digital-to-analog converter(CDAC) configured to sample the first MAC output voltage through thefirst sampling switch to provide a sampled first MAC output voltage; anda second capacitive digital-to-analog converter (CDAC) configured tosample the second MAC output voltage through the second sampling switchto provide a sampled second MAC output voltage; an averaging switchcoupled between the first CDAC and the second CDAC, wherein theaveraging switch is configured to close to average the sampled first MACoutput voltage with the sampled second MAC output voltage to provide anaveraged MAC output voltage; and a comparator configured to compare theaveraged MAC output voltage to a reference voltage to provide acomparator output signal.
 19. The system of claim 18, wherein thereference voltage is approximately one-half a power supply voltage. 20.The system of claim 18, wherein the first CDAC is further configured tosubtract a bias voltage from the averaged voltage, the analog-to-digitalconverter further comprising a control logic circuit configured tocontrol the bias voltage.
 21. The system of claim 18, wherein the firstCDAC and the second CDAC are 3-bit CDACs.
 22. The system of claim 18,wherein the MAC circuit includes a read bit line, and wherein eachcompute-in-memory bitcell includes: a pair of cross-coupled invertershaving a first output node for a weight bit; a word line having avoltage responsive to an input vector bit; a capacitor having a firstplate connected to the read bit line; and a first pass transistorconnected between the first output node and a second plate of thecapacitor and having a gate connected to the word line.
 23. The systemof claim 22, wherein each first pass transistor comprises a p-typemetal-oxide-semiconductor (PMOS) transistor.
 24. The system of claim 22,wherein each compute-in-memory bitcell further comprises: a read wordline; and a transistor coupled between the second plate of the capacitorand ground and having a gate connected to the read word line.
 25. Thesystem of claim 22, wherein the MAC circuit is configured to pre-chargethe read bit line to a power supply voltage.